Interleaving 3D model feature prediction and matching to support multi-sensor object recognition

The object recognition system presented combines on-line feature prediction with an iterative multisen-sor matching algorithm. Matching begins with an initial object type and pose hypothesis. An iterative generate-and-test procedure then reenes the pose as well as the sensor-to-sensor registration for separate range and electro optical sensors. During matching, object features predicted to be visible are updated to reeect changes in hypothesized object pose and sensor registration. The match found is locally optimal in terms of the complete space of possible matches and globally consistent in the sense of preserving the 3D constraints implied by sensor and object geometry. Results on real data are presented which demonstrate the algorithm correcting for up to 30 errors in initial orientation and 5m errors in initial translation.